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An image searching system and image searching method, and a 
recording medium storing an image searching program 

BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates generally to an 
image searching system for finding an image that resembles 
a specified image from among images managed by the image 
searching system where the search criteria are determined 
from the specified image. The present invention relates 
more specifically to an image searching system for finding 
an image based on a plurality of specified images* 

Description of the Related Art 

Images stored to a database for managing image 
data are typically stored with additional information that 
can be used to specify search conditions, including search 
keys, such as identifying keywords and image color keys, 
and image features, such as information on the shapes and 
colors appearing in the image. Using this common type of 
image database, it is possible to search the stored images 
to find an image similar to a user-specified image. 

Image similarity in similar- image searching is 
determined based on such feature quantities as image shape, 
texture, and color. More specifically, the user specifies 



an image in the image database that is similar to a desired 
image to be searched for. Feature quantities are then 
extracted from the image selected by the user as the search 
key (referred to below as the "key image"), and the 
extracted feature quantities are then compared with the 
feature quantities registered in the image database. Image 
similarity indicative of the similarity between the key 
image and a found image is then calculated. Information on 
any stored image determined to resemble the key image based 
on this calculated image similarity is then returned as the 
search result. 

Usually there is only one key image selected by 
the user, and it is often impossible to find the one image 
sought by the user based on the feature quantities defined 
from a single key image. Search accuracy is therefore often 
low. This can happen, for example, when there are plural 
features in a single image and the user focuses on only one 
feature, that is, when the user's search criteria are 
narrower than the criteria that can be extracted from the 
key image, as well as when numerous search conditions are 
specified to broaden the range of the search. 

Japanese Patent Laid-Open Publication 7-65169 
teaches one method proposing to solve this problem. 
According to the taught method, a plurality of key images 
is specified, a specific area in each of the key images is 



selected, and the features to be used as search criteria 
are extracted from the selected areas. The search is then 
conducted using these features from plural selected areas 
in plural images. This method thus makes it possible to 
search for images having all of the features found in the 
specified areas of the plural key images. In other words, 
this method enables the user to specify a wide range of 
search conditions based on plural images. 

The drawback to said taught method is that the 
user is required to specify in each of the selected key 
images those features that are to be used for searching, 
and this task becomes increasingly difficult and time- 
consuming as the number of key images increases and the 
variety of selected features increases. 

The present invention is therefore directed to an 
image searching system and an image searching method for 
reducing the user burden and improving search accuracy when 
a plurality of key images is specified and a desired image 
is searched for based on these selected key images. 

The present invention is further directed to a 
recording medium for storing an image searching program 
embodying the image searching method of the present 
invention. 



SUMMARY OF THE INVENTION 
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A first image searching system according to the 
present invention comprises: an image storage means for 
storing a plurality of target images for searching; a 
selection means for selecting a plurality of key images for 
determining the search criteria; a first feature quantity 
extraction means for extracting one or a plurality of 
feature quantities from each of the plurality of key images 
selected by the selection means; a second feature quantity 
extraction means for extracting one or a plurality of 
feature quantities from a target image; a feature quantity 
identification means for comparing a feature quantity 
extracted by the first feature quantity extraction means 
with the same type of feature quantity in each of the 
plurality of key images to determine a feature quantity to 
be used for searching; a degree of similarity calculating 
means for calculating a degree of similarity between a 
target image and key image using the feature quantity 
identified by the feature quantity identification means for 
searching; and an extraction means for extracting as an 
image similar to a key image any target image for which the 
degree of similarity calculated by the degree of similarity 
calculating means exceeds a specific value. 

Thus comprised, this first image searching system 
compares features extracted from plural key images to 
determine the features to be used as search criteria, and 
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then calculates image similarity using the identified 
features (feature quantities) . 

In this first image searching system, the feature 
quantity identification means preferably defines as a 
feature quantity type to be used for searching a feature 
quantity type having an approximately equal value in each 
of the compared key images. 

A second image searching system according to the 
present invention comprises: an image storage means for 
storing a plurality of target images for searching; a 
selection means for selecting a plurality of key images for 
determining the search criteria; a first feature quantity 
extraction means for extracting a feature quantity from 
each of the plurality of key images selected by the 
selection means; a second feature quantity extraction means 
for extracting a feature quantity from a target image; a 
first degree of similarity calculating means for 
calculating a first degree of similarity indicative of the 
similarity between a target image and all of the plurality 
of key images using the feature quantities extracted by the 
first and second feature quantity extraction means; a 
second degree of similarity calculating means for 
calculating a second degree of similarity indicative of the 
similarity between a target image and at least one of the 
plurality of key images using the feature quantities 
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extracted by the first and second feature quantity 
extraction means; a third degree of similarity calculating 
means for calculating a third degree of similarity from the 
first and second degrees of similarity where the first 

5 degree of similarity is weighted greater than the second 
degree of similarity; and an extraction means for 
extracting as an image similar to a key image any target 
image for which the third degree of similarity calculated 
by the third degree of similarity calculating means exceeds 

0 a specific value. The third degree of similarity in this 
case is calculated with greater weight assigned to the 
first degree of similarity than to the second degree of 
similarity 

A first image searching method according to the 
5 present invention searches in a plurality of stored target 
images for an image that is similar to a key image based on 
the key image. This first image searching method comprises: 
a selection step for selecting a plurality of key images; a 
first feature quantity extraction step for extracting one 
0 or a plurality of feature quantity types from each of the 
plurality of selected key images; a second feature quantity 
extraction step for extracting one or a plurality of 
feature quantity types from the target images; a feature 
quantity identification step for comparing a feature 
5 quantity extracted by the first feature quantity extraction 



step with the same type of feature quantity in each of the 
plurality of key images to determine a feature quantity 
type to be used for searching; a degree of similarity 
calculating step for calculating a degree of similarity 
between a target image and key image using the feature 
quantity type identified for searching; and an extraction 
step for extracting as an image similar to a key image any 
target image for which the calculated degree of similarity 
exceeds a specific value. 

A second image searching method for searching in 
a plurality of stored target images for an image that is 
similar to a key image based on said key image comprises: a 
selection step for selecting a plurality of key images; a 
first feature quantity extraction step for extracting a 
feature quantity from each of the plurality of selected key 
images; a second feature quantity extraction step for 
extracting a feature quantity from a target image; a first 
degree of similarity calculating step for calculating a 
first degree of similarity indicative of the similarity 
between a target image and all of the plurality of key 
images using the feature quantities extracted by the first 
and second feature quantity extraction steps; a second 
degree of similarity calculating means for calculating a 
second degree of similarity indicative of the similarity 
between a target image and at least one of the plurality of 
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key images using the feature quantities extracted by the 
first and second feature quantity extraction steps; a third 
degree of similarity calculating step for calculating a 
third degree of similarity from the first and second 
degrees of similarity where the first degree of similarity 
is weighted greater than the second degree of similarity; 
and an extraction step for extracting as an image similar 
to a key image any target image for which the third degree 
of similarity exceeds a specific value. 

A first recording medium according to the present 
invention is a computer-readable recording medium for 
recording an image searching program where the image 
searching program embodies a method for searching in a 
plurality of stored target images for an image that is 
similar to a key image based on said key image. Specific 
steps in this image searching program include: a selection 
step for selecting a plurality of key images; a first 
feature quantity extraction step for extracting one or a 
plurality of feature quantity types from each of the 
plurality of selected key images; a second feature quantity 
extraction step for extracting one or a plurality of 
feature quantity types from the target images; a feature 
quantity identification step for comparing a feature 
quantity extracted by the first feature quantity extraction 
step with the same type of feature quantity in each of the 
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plurality of key images to determine a feature quantity 
type to be used for searching; a degree of similarity 
calculating step for calculating a degree of similarity 
between a target image and key image using the feature 
quantity type identified for searching; and an extraction 
step for extracting as an image similar to a key image any 
target image for which the calculated degree of similarity 
exceeds a specific value. 

A second recording medium according to the 
present invention is also a computer-readable recording 
medium for recording an image searching program. Specific 
steps in this image searching program include: a selection 
step for selecting a plurality of key images; a first 
feature quantity extraction step for extracting a feature 
quantity from each of the plurality of selected key images; 
a second feature quantity extraction step for extracting a 
feature quantity from a target image; a first degree of 
similarity calculating step for calculating a first degree 
of similarity indicative of the similarity between a target 
image and all of the plurality of key images using the 
feature quantities extracted by the first and second 
feature quantity extraction steps; a second degree of 
similarity calculating means for calculating a second 
degree of similarity indicative of the similarity between a 
target image and at least one of the plurality of key 



images using the feature quantities extracted by the first 
and second feature quantity extraction steps; a second 
degree of similarity calculating means for calculating a 
second degree of similarity indicative of the similarity 
between a target image and at least one of the plurality of 
key images using the feature quantities extracted by the 
first and second feature quantity extraction steps; a third 
degree of similarity calculating step for calculating a 
third degree of similarity from the first and second 
degrees of similarity where the first degree of similarity 
is weighted greater than the second degree of similarity; 
and an extraction step for extracting as an image similar 
to a key image any target image for which the third degree 
of similarity exceeds a specific value. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other objects and features of the 
present invention will be readily understood from the 
following detailed description taken in conjunction with 
preferred embodiments thereof with reference to the 
accompanying drawings, in which like parts are designated 
by like reference numerals and in which: 

Fig. 1 is a typical overview of an image 
searching system according to a preferred embodiment of the 
present invention; 
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Fig, 2 is a block diagram of primarily the 
control device of the image searching system shown in Fig. 
1; 

Fig. 3 shows a typical image database table 

5 structure; 

Fig. 4 is a flow chart of the main loop of the 
image searching method embodied in the image searching 
system shown in Fig. 1; 

Fig. 5 is a flow chart of the image registration 
10 process shown as step S3 in Fig. 4; 

Fig. 6 is a flow chart of the similar image 
searching process shown as step S4 in Fig. 4; 

Fig. 7 is a flow chart of the search feature 
compilation process shown as step S43 in Fig. 6; 
15 Fi g- 8 is a data table used to describe a method 

of calculating common features; 

Fig. 9 is a data table of sample feature 
quantities extracted from a key image; 

Fig. 10 is a data table of sample feature 
20 quantities extracted from a target image; 

Fig. 11 is a data table used to describe 
similarity calculation based on common feature quantities; 

Fig. 12 is a flowchart of the similar image 
searching process shown as step S4 in Fig. 4, that is 
25 executed according to a second preferred embodiment of the 
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present invention; and 

Fig. 13 is a flowchart of the search feature 
compilation process shown as step S43 in Fig. 6, that is 
executed according to the second preferred embodiment of 
5 the present invention. 

Fig. 14 is a data table used to describe 
similarity calculation based on all feature quantities; 

Fig. 15 is a data table used to describe 
similarity calculation based on a degree of similarity 
0 based on common feature quantities and a degree of 
similarity based on all feature quantities. 



DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The preferred embodiments of an image searching 
system according to the present invention are described 
below with reference to the accompanying figures. An image 
searching system according to the present invention 
searches for an image similar to common features found in a 
plurality of specified key images. The image searching 
0 system extracts one or a plurality of feature quantities 
from the plural key images and compares similar feature 
quantities in the key images to determine the types of 
feature quantities to use for searching. The degree of 
similarity is then calculated using the specified feature 
5 quantity types, and an image search is then conducted based 
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on this degree of similarity. It is therefore possible to 
search for images having features common to a plurality of 
key images* 

The image searching system of the present 
5 invention can alternatively calculate the degree of 
similarity using both a degree of similarity indicative of 
the resemblance to all of the plural key images, and a 
degree of similarity indicative of the resemblance to one 
or some larger subset of the plural key images . This method 
0 makes it possible to expand the range of the search, and 
thus improve search accuracy. 



Overall configuration of the image searching system 

Fig. 1 shows the basic configuration of an image 

5 searching system, referred to as the "system" below, 
according to a preferred embodiment of the present 
invention. As will be known from Fig. 1, this system is 
built around a control device 1 for controlling the overall 
system and comprising a central processing unit (CPU) . 

0 Connected to this exemplary control device 1 are: a display 
2 for displaying text and images, as well as control 
prompts and other interface elements; a keyboard 3 and 
mouse 4 for data entry and operating the system; a floppy 
disk drive 5b in which a floppy disk 5a can be used and a 

5 hard disk drive 6 for reading and writing data to a data 
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storage medium; a printer 7 for printing text and images to 
hard copy; a scanner 8 for capturing image data; a CD-ROM 
drive 9b for accessing data stored to a CD-ROM 9a; a 
speaker 10 for audio output; and a microphone 11 for audio 
5 input . 

Fig. 2 is a block diagram of the image searching 
system shown in Fig. 1. 

Connected directly to the CPU 201 is a clock 202 
for generating the reference clock required to regulate 

0 system operations. A ROM 203 for storing an application 
program for controlling the image searching system, and a 
RAM 204 for temporarily storing data and programs used by 
the CPU 201 for system control, are connected to the CPU 
201 via a data bus 220. Circuits connected to the CPU 201 

5 via the data bus 220 include: a display control circuit 205 
for controlling the display 2 to display text and graphics; 
a keyboard control circuit 206 for controlling transfer of 
input from the keyboard 3; a mouse control circuit 207 for 
controlling transfer of input from the mouse 4; a floppy 

0 disk drive control circuit 208 for controlling the floppy 
disk drive 5b; a hard disk drive control circuit 209 for 
controlling the hard disk drive 6; a printer control 
circuit 210 for controlling output to the printer 7; a 
scanner control circuit 211 for controlling the scanner 8; 

5 a CD-ROM drive control circuit 212 for controlling the CD- 
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ROM drive 9b; a speaker control circuit 213 for controlling 
the speaker 10; and a microphone control circuit 214 for 
controlling the microphone 11. An expansion slot 215 for 
connecting different types of system expansion boards is 

5 also connected via the data bus 220. 

It should be noted that a SCSI board, for example, 
can be connected to the expansion slot 215 and used for 
connecting a floppy disk drive 5b, hard disk drive 6, 
scanner 8, CD-ROM drive 9b, or other device. 

0 It should also be noted that while a floppy disk 

5a and hard disk drive 6 are the preferred image data 
storage means in this exemplary system, the invention shall 
obviously not be limited thereto as any other data storage 
medium that the system can use can be used for image data 

5 storage, including magneto-optical (MO) disks [and DVD 
disks] . 

Furthermore, while a scanner 8 is used in this 
exemplary system for image data capture and input, other 
data input devices can also be used, including video still 
0 cameras and digital cameras. 

In addition, while a printer 7 is used as an 
output device, a digital photocopier or other type of hard 
copy output device can also be used. 

A program embodying the control method of the 
5 present invention is also stored to ROM 203 in this 
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preferred embodiment. This control program, however, can be 
alternatively stored in whole or in part to another data 
storage medium, including a floppy disk 5a, hard disk drive 
6, or CD-ROM 9a. In this case, the required data and 
5 program content can be read from the data storage medium as 
needed, buffered to RAM 204, and executed from RAM 204. 

Image database 

To store and manage image data, an image 

0 searching system according to this preferred embodiment 
also has an image database and a color space table. The 
image database contains both the image data and additional 
information used for the search keys. The color space table 
is referenced when extracting color features from an image. 

5 Both the image database and color space table are logic 
objects stored on the hard disk drive 6 or other data 
storage medium. 

An exemplary image database 50 in the present 
system is shown in Fig. 3. As will be known from Fig. 3, 

0 this image database 50 manages the image data to be 
searched, and various columns of image attributes that are 
used as search parameters for finding desired image data. 
These image attributes include keywords, a color key, and 
other search keys, and various feature quantities 

5 indicative of specific features in the image data. More 
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specifically, these feature quantities include a color 
feature indicative of features relating to image color, a 
shape feature indicative of shapes in the image, and a 
texture feature indicative of texture patterns. The 
5 similarity between images is calculated based on this 
feature information . 

Controlling the image searching system 

A specific method of controlling the above- 
CD described image searching system is described next below 
with reference to the accompanying flow charts. 

Main loop of the control program 

Fig. 4 is a flow chart of the main loop of a 

5 control program run by the CPU 201 in the image searching 
system shown in Fig. 1. 

The procedure starts by initializing the various 
flags and variables used in the rest of the procedure, and 
to display the default menus (SI) . An exemplary default 

0 menu screen displays various icons representing and used 
for selecting specific processes such that when a user 
selects one of the icons, the corresponding process is 
executed. It should be further noted that the system of 
this preferred embodiment enables the operator to select 

5 various processes and enter information using the keyboard 
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3 and mouse 4 from the default menu screen and other 
screens that are displayed on the display 2. After 
initialization in step SI, decision diamond S2 detects 
whether the user has selected a menu item from the default 
menu screen (S2) . 

If in step S2 the user selects "register image , " 
for example, the image registration process (S3) is 
selected and run, and the procedure then advances to step 
S6. This image registration process (S3) is run to store 
image data and related feature information in the image 
database 50, and is described in further detail below with 
reference to the flow chart in Fig. 5. 

If in step S2 the user selects "search for 
similar image, " for example, the similar image searching 
process (S4) is selected and run, and the procedure then 
advances to step S6. This similar image searching process 
(S4) is run to search in the image database 50 for an image 
similar to a selected key image, and is described in 
further detail below with reference to the flow chart in 
Fig. 6. 

If in step S2 the user selects another menu item, 
the corresponding process (S5) is selected and run, and the 
procedure then advances to step S6. 

If no menu item is selected in S2, the procedure 
advances directly to step S6. After the procedure defined 
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for step S6 is run and completed, the main loop returns to 
step S2 and repeats. 

It should be noted that the tasks that might be 
performed as the "other menu process" selected in S5 are 
5 not unique to or directly related to the present invention, 
and further description thereof is thus omitted below. The 
image registration process (S3) and the similar image 
searching process (S4) of the present invention are 
described next in detail below. 

10 

Image registration process 

The image registration process shown as step S3 
in Fig. 4 is described next in detail below with reference 
to the flow chart in Fig. 5. This process extracts feature 

15 quantities from a user-specified image to be stored in the 
image database, and then stores the specified image and 
extracted features to the image database 50. 

This routine starts by reading the user-specified 
image to be stored to the image database 50 (S31) . A 

20 feature extraction process is then run (S32) to extract 
feature quantities from the specified image. In this 
preferred embodiment of the invention, the extracted 
feature quantities include color, shape, and texture, but 
the invention shall obviously not be limited thereto. The 

25 extracted feature quantities are then related* to the image 
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data of the specified image (S33) , and finally registered 
in the image database 50 together with the related 
specified image data (S34) . A file name, image size 
information, and other image attributes are also normally 
stored with or related to the image data. The procedure 
then returns to the main loop (step S6 in Fig. 4) . 
[* NOTE: I f ve used "related" here because the database 
could be a flat file or relational database, and "related" 
will cover the association between feature quantities and 
image data for both database types (the tables in the 
illustrations seem to imply a flat-file (non-relational) 
database structure) . ] 

Similar image searching processing 

(a) Similar image search based on common feature 
quantities (first embodiment) 

According to first embodiment, common feature 
quantities are extracted from a plurality of key images. 
Upon evaluating a similarity of a target image to a key 
image, only these common feature quantities are used for 
calculating a degree of similarity of the target image to a 
key image. In other words, feature quantities other than 
the common feature quantities are not used for calculation 
of the degree of similarity. 

The similar image searching processing according 



to first embodiment (shown as step 4 in Fig. 4) is 
described below with reference to a flow chart shown in Fig. 
6. In the present processing, predetermined feature 
quantities are extracted from each of plural key images 
specified by a user and feature quantities indicative of 
features common to the plural key images are selected as 
common feature quantities among the extracted feature 
quantities. Then, the searching processing is performed to 
find out an image similar to a key image among images 
registered in the image database 50 based on the common 
feature quantities . 

Referring to Fig. 6, at first, a plurality of key 
images specified by a user are read (step S41) . Feature 
quantities of each key image are extracted from the image 
database 50 (step S42) . In this step, feature quantities 
of plural kinds such as color, shape, texture and so on are 
obtained from each key image. Next, a search feature 
quantity compilation processing is performed to calculate 
common feature quantities for searching a similar image by 
comparing the feature quantities of respective key images 
with each other (step S43) . This search feature quantity 
compilation processing will be described later. 

Thereafter, to find an image among images 
registered in the image database 50 which is similar to the 
key images, the common feature quantities calculated in 
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step S43 are compared with feature quantities of each image 
registered in the image database, and the degree of 
similarity between the key images and the presently 
selected image is calculated (from step S44 to step S46) . 

Namely, it is determined if there is an image in 
the image database 50 for which the degree of similarity to 
the key images has not yet been calculated (herein below 
referred to as a "target image") (step S44) . If such a 
target image is present in the image database 50, the 
feature quantities for that target image are read from the 
image database 50 (step S45) . Then, a degree of similarity 
indicative of a similarity between the target image and 
each key image is calculated based on the common feature 
quantities of the key images and the feature quantities of 
the key images and the feature quantities of the target 
image (step S46) and the processing loops back to step S43. 
This calculation processing of the degree of similarity 
will be described later. These steps S43 to S45 are 
repeated until the similarity calculation has been 
completed for all target images in the image database. 

When the degree of similarity has been calculated 
for all target images in the image database 50, an output 
processing of search results is performed (step S47) and 
the processing returns to the main routine. In the output 
processing of the search results, it is determined that 
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those target images for which the calculated degree of 
similarity exceeds a predetermined threshold value are 
similar to the key images. The target images determined to 
be similar are then displayed on screen in turn from the 
highest degree of similarity. 

Alternately, without providing any threshold for 
determination of the similarity, a predetermined number of 
target images can be displayed in a descending order from 
the highest degree of similarity. Further, the search 
results can be displayed by simply outputting the file name 
of the target image data, preferably with related 
attributes such as its location in the database . The 
target images tested by the search loop from step S44 to 
step S46 shall* also not be limited to all of the images 
stored in the image database 50. For instance, only images 
preselected based on predetermined conditions can be tested. 

The search feature quantity compilation 
processing (step S43 in Fig. 6) is described next with 
reference to a flow chart shown in Fig. 7. This processing 
calculates the common feature quantities to be used upon 
searching a target image which is similar in common to a 
plurality of key images. That is, the common feature 
quantities indicative of features in common to a plurality 
of key images are obtained by comparing feature quantities 
of the same kind among the plurality of key images and 
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choosing feature quantities of the same kind having values 
being approximate to each other as the common feature 
quantities. This processing is described in further detail 
below. 

As shown in Fig, 7, the processing starts by 
calculating, for every kind of the feature quantity, 
differences among the feature quantities of the key images 
(step S431) . Next, a mean value of the differences 
obtained in step S431 is calculated for every kind of the 
feature quantity (S432) . An example of this calculation is 
shown in Fig. 8. This figure shows an example obtained for 
three key images 1 to 3 each having five feature quantities 
1 to 5 as shown in Fig. 9. The mean value of the 
differences with respect to the same feature quantity 
indicates the smaller the mean value the greater the image 
similarity with respect to that feature quantity. 

Thereafter, every feature quantity or the kind of 
the feature quantity having a mean value equal to or less 
than a predetermined threshold is selected as a common 
feature quantity (step S433) . That is, when the mean 
feature quantity difference is less than the threshold 
value, that feature quantity is determined to be common to 
the plural key images. In the example shown in Fig. 8, 
this threshold value is set to 0.1 and feature quantity 2 
and feature quantity 3 are thus selected as common feature 



quantities. Thus, the feature quantities or kinds of 
feature quantities to be used for searching are determined 
by comparing feature quantities of the same kind in the key 
images. In the last step, a specific flag is set for every 
feature quantity selected as a common feature quantity 
(step S434) . 

Next, the processing for calculating the degree 
of similarity (step S46 in Fig. 6) is described below. 
Based on the common feature quantities extracted from the 
key images and a target image, this processing calculates 
the degree of similarity of the target image to all key 
images or either one of chern. 

According to the present embodiment, only feature 
quantities selected as common feature quantities in step 
S43 are used for calculating the degree of similarity of 
the target image to the key image. 

Accordingly, feature quantities other than common 
feature quantities are not used for the calculation of the 
degree of similarity. Due to this, images having features 
common to plural key images can be obtained as the results 
of search with a high accuracy. In this case, the target 
image similar to all of the plural key images. In other 
words, this search method imposes a search under an AND 
condition among the key images and thereby, enables to 
restrict ranges of searching conditions to reasonable ones. 



Fig. 11 shows an example of the calculation for 
the degree of similarity. In this example, the distance 
between each key image and the target image .is calculated 
based on the values of feature quantity 2 and feature 
quantity 3 selected as the common feature quantities and 
the degree of similarity is obtained based on the 
calculated distances. The values shown in Fig. 11 are 
obtained from the following equations (1) to (3) . 

Difference between feature quantities = | (key image 
feature quantity) - (target image feature quantity)] (1) 

Distance = square root of sum of (each feature 
quantity difference ) 2 (2) 

Degree of similarity = 1.0/distance (3) 

The degree of similarity thus obtained indicates 
that the image similarity becomes high as the calculated 
value of the degree of similarity becomes large, 
(b) Similar image searching based on all kinds of feature 
quantities (second embodiment) 

According to second embodiment, the degree of 
similarity between a target image and key images is 
calculated for each of the key images based on feature 
quantities of all kinds. From the degrees of similarity 
calculated for each key image, the highest degree of 
similarity is then selected as the degree of similarity of 
the target image to the corresponding key image. In this 



case, images similar to at least one of the plural key 
images are searched. That is, the image search is 
performed under an OR logic condition and, accordingly, it 
becomes possible to broaden the ranges of searching 
conditions by increasing the number of key images. Second 
embodiment performs a processing shown in Fig. 12 and Fig. 
13 instead of Figs. 6 and 7 of first embodiment. 

Referring to Fig. 12, a plurality of key images 
specified by a user are read at first (S141) . Feature 
quantities are obtained from the image database 50 for each 
key image (S142) . Namely, feature quantities of plural 
kinds such as color, shape, texture and the like are 
obtained form each key image. 

Then, the feature quantities of each key image 
obtained in step S142 are compared with feature quantities 
of each target image registered in the image database 50, 
the similarity between each key image and the target image 
is determined and thereby, target images being similar to 
respective key images are searched (steps from S143 to 
S145) . 

At first, it is determined whether or not there 
is a target image in the image database for which the 
degree of similarity to each key image has not yet 
calculated (step S143) . If such a target image is present 
in the image database 50, the feature quantities for that 
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target image are read from the image database (step S144) . 
Then, calculation of the degree of similarity indicative of 
similarity between images is performed based on the feature 
quantities of each key image and those of the target image 
(step S144). After this calculation, the processing loops 
back to step S143. This calculation processing of the 
degree of similarity (step S145) will be described later. 
These steps S143 to S145 are repeated until the similarity 
calculation has been completed for all target images in the 
image database 50. 

When the degree of similarity has been calculated 
for all target images in the image database 50, an output 
processing of search results is performed (step S147) and 
then, the processing returns to the main routine. In this 
output processing of search results (step S146) , based on 
the degrees of similarity calculated for all target images 
in the image database 50, the key image having a degree of 
similarity higher than a predetermined threshold value is 
determined to be similar to the corresponding key image. 
The target images determined to be similar are then display 
on screen in sequence from the highest degree of similarity. 

Alternately, without providing any threshold for 
determination of the similarity, a predetermined number of 
target images can be displayed in a descending order from 
the highest degree of similarity. Further, the search 



results can be displayed by simply outputting the file name 
of the target image data, preferably with related 
attributes such as its location in the database. The 
target images tested by the search loop from step S144 to 
step S146 shall also not be limited to all of the images 
stored in the image database 50. For instance, only images 
preselected based on predetermined conditions can be tested. 

Next, the calculation processing of degree of 
similarity is described below with reference to a flow 
chart shown in Fig. 13. 

As shown in Fig. 14, a distance between a feature 
quantity of the key image and that of the same kind of the 
target image is calculated for each of all feature 
quantities, and a distance between the key images and the 
target image is calculated based on the distances of 
respective feature quantities. 

Based on the distance between images, the degree 
of similarity is calculated (step S1451) . More 
specifically, the values shown in Fig. 14 are obtained from 
the following equations (4) to (6) . 

Feature quantity distance = | (key image feature 
quantity) - (target image feature quantity) | (4) 

Distance = sqrt of {sum of (each feature quantity 
distance) 2 } (5) 

Degree of similarity = 1.0/distance 
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Note that with this method image similarity 
increases as the calculated degree of similarity value 
rises. Because the degree of similarity to key image 3 is 
highest in the example shown in Fig. 14, that value is used 
as the degree of similarity between the current target 
image and the key images (step S1452) . 

The output processing of search results mentioned 
above with respect to step S14 6 is performed based on the 
degrees of similarity determined in step S1452. 

(c) Similar image searching processing using degrees 
of similarity based on all feature quantities and degrees 
of similarity based on common feature quantities (third 
embodiment) 

This method combines the degrees of similarity 
based on the common feature quantities calculated according 
to first embodiment (a) and those based on all feature 
quantities calculated according to second embodiment (b) , 
to obtain final degrees of similarity. In this case, the 
degree of similarity based on common feature quantities is 
weighted and the weighted degree of similarity and the 
degree of similarity based on all feature quantities are 
added to obtain a mean value thereof. This mean value is 
assigned as the degree of similarity of the target image to 
the key image. Thus, this method performs a similar image 
search in combination of AND and OR conditions. As a 



result, it becomes possible to find an image having 
features common to all key images among images similar to 
any one of the key images and therefore, to improve the 
search accuracy compared with second embodiment (b) . 

An example of the calculation of the degree of 
similarity is shown in Fig. 15. In this example, the 
weight for the degree of similarity a calculated in Fig. 11 
is set larger than that for the degree of similarity 
calculated in Fig. 14. A weighted mean value is than 
calculated to obtain the final degree of similarity c. 
This final degree of similarity c is obtained according to 
the next equation (7) . 

Degree of similarity = { (degree of similarity a x 
weight wa)+ (degree of similarity b x weight wb) } / (wa+wb) 

(7) 

As stated above, according to the present 
embodiment, feature quantities are extracted from a 
plurality of key images, when specified and those common to 
the plural key images are selected to determine similarity 
between images using these common feature quantities. Thus, 
the common feature quantities are automatically determined 
from among plural key images and accordingly, the user 
burden can be reduced. It is also possible to search for 
images having features common to plural key images. This 
enables to narrow the ranges of respective searching 
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conditions by specifying plural key images. Also, the 
search can be similarly expanded to broader ranges of 
searching conditions by considering the degree of 
similarity indicating similarity of an image to at least 
one of plural key images. Search accuracy can be thus 
improved. 

As will be known from the above descriptions of 
the present invention, a first image searching system, a 
first image searching method, and a first recording medium 
storing an image searching program embodying the first 
image searching method of the invention determines the 
feature quantities to be used for image searching based on 
the feature quantities extracted from one or more key 
images, and calculates a degree of similarity using only 
the feature quantities selected as search criteria. It is 
therefore possible to search for images with features 
common to those of the selected key images. The search 
conditions can thus be restricted by specifying plural key 
images, and search accuracy can thus be improved. The user 
burden is also reduced because feature quantities are 
automatically extracted from the specified key images, and 
similar images are searched for based on the extracted 
feature quantities . 

As will be known from the above descriptions of 
the present invention, a second image searching system, a 
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second image searching method, and a second recording 
medium storing an image searching program embodying the 
second image searching method of the invention calculates 
the degree of similarity used for searching by combining a 
degree of similarity indicative of similarity to all of the 
plurality of key image, and a degree of similarity 
indicative of similarity to any one of the plurality of key 
image. It is therefore possible to set a wide range of 
search criteria by selecting a plurality of key images, and 
further improve the accuracy of similar image searches. 

Although the present invention has been described 
in connection with the preferred embodiments thereof with 
reference to the accompanying drawings, it is to be noted 
that various changes and modifications will be apparent to 
those skilled in the art. Such changes and modifications 
are to be understood as included within the scope of the 
present invention as defined by the appended claims, unless 
they depart therefrom. 
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What is claimed is: 

1* An image searching system comprising: 

an image database storing a plurality of database 
5 images to be searched for; a specifying controller for 
specifying a plurality of key images used to specify search 
conditions; 

an extracting controller for extracting common feature 
values of common images from the plural key images 
0 specified by the specifying controller; 

a calculating controller for comparing the common 
feature values, extracted by the extracting controller, 
with the feature values of the plural database images to 
thereby sequentially calculate similarities between the 
5 common feature values and the database image feature 
values; and 

a searching controller for retrieving from the 
database images one of the images which is similar to the 
key image, based on a similarity calculated by the 
0 calculating controller. 

2 * Th e system as claimed in Claim 1, wherein the 

extracting controller includes: 

an extracting means for extracting a plurality of 
types of the feature quantities from the respective key 
5 images specified by the specifying controller; 
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a selecting means for comparing the feature quantities, 
extracted by the extracting means, among the plural key 
images specified by the specifying controller to thereby 
select at least one of the types of the feature quantities; 
5 and 

a determining means for determining the common feature 
quantities based on the at least one type of the features 
quantities selected by the selecting means. 

3 - The system as claimed in Claim 2, wherein the 
10 selecting means is operable to compare the feature 

quantities of the same types among the plural key images 
specified by the specifying controller and wherein the 
determining means is operable to calculate an average value 
of the feature quantities of the plural key image with 
15 respect to the types of the feature quantities selected by 
the selecting means, to thereby determine the calculated 
average value as representing the common feature quantities. 

4- An image searching system which comprises: 

an image database storing a plurality of database 
20 images to be searched for; 

a specifying controller for specifying a plurality of 
key images used to specify search conditions; 

a calculating controller for comparing the plural 
images, specified by the specifying controller, with the 
25 plural database images to thereby calculate similarities 
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between the common feature values and the database image 
feature values; 

a selecting controller for retrieving a particular key 
image from the specified images based on the similarities 
5 calculated by the calculating controller; and 

a searching controller for retrieving the images from 
the database images based on the similarity between the key 
image, selected by the selecting controller, and the 
database images. 

10 5 * The system as claimed in Claim 4, wherein the 

selecting controller is operable to select as a particular 
one of the plural specified key images, the key images 
which most resemble to the database images being searched 
for . 

15 6 - The system as claimed in Claim 5, wherein the 

calculating controller is operable to calculate a plurality 
of types of the feature quantities from the plural key 
images and then to calculate a degree of similarity by 
comparing the feature quantities with the database images 

20 for each type, and wherein the selecting controller selects, 
as the particular key image from the plural specified 
images, the key images which most resemble to the database 
images being searched with respect to an average value of 
degrees of similarities calculated by the calculating means 

25 for each type of the feature quantities. 
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7 - An image searching system which comprises: 

an image database storing a plurality of database 
images to be searched for; 

a specifying controller for specifying a plurality of 

5 key images used to specify search conditions; 

a first calculating controller for comparing all of 
the key images, specified by the specifying controller, 
with respective feature values of the database images to 
thereby calculate similarities therebetween; 

0 a second calculating controller for selecting a 

particular key image from the plural key images specified 
by the specifying controller and for comparing the 
particular key image with the database images to thereby 
calculate similarities therebetween; 

5 a third calculating controller for calculating a final 

similarity for use in searching based on the similarities 
calculated respectively by the first and second calculating 
controllers; and 

a searching controller for retrieving one of the 

0 database images, which is similar to the particular key 
image, based on the final similarity calculated by the 
third calculating controller. 

8 - Th e image searching system as claimed in Claim 7, 
wherein the third calculating controller is operable to 

5 increase a weight of the degree of similarity, calculated 
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by the first calculating controller, to a value greater 
than that of the degree of similarity, calculated by the 
second calculating controller, to thereby calculate the 
final degree of similarity. 

5 9 - The image searching system as claimed in claim 8, 

wherein the first calculating controller is operable to 
extract the common feature quantities of the image common 
to all of the key images, and to compare those common 
feature quantities with the database image to thereby 

0 calculate the degree of similarity. 

10. The image searching system as claimed in Claim 9, 
wherein the second calculating controller is operable to 
select the key images most similar to the database image 
from the key images and to calculate the degree of 

5 similarity. 

11. An image searching method which comprises the 
steps of: 

storing a plurality of database images to be searched 
for in a database; 
0 specifying a plurality of key images used to specify 

search conditions by means of a specifying controller; 

extracting by means of an extracting controller, 
common feature values of common images from the plural key 
images specified by the specifying controller; 
5 comparing by means of a calculating controller, the 
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common feature values, extracted by the extracting 
controller, with the feature values of the plural database 
images to thereby sequentially calculate similarities 
between the common feature values and the database image 

5 feature values; and 

retrieving from the database images one of the images 
which is similar to the key image, based on a similarity 
calculated by the calculating controller, by means of a 
searching controller. 

0 12. The method as claimed in Claim 11, wherein the 

extracting step includes the sub-steps of extracting a 
plurality of types of the feature quantities from the 
respective key images specified by the specifying 
controller; comparing by means of a selecting means the 

5 feature quantities, extracted by the extracting sub-step, 
among the plural key images specified by the specifying 
controller to thereby select at least one of the types of 
the feature quantities; and determining the common feature 
quantities based on the at least one type of the features 

0 quantities selected by the selecting means. 

13 • Th e method as claimed in Claim 12, wherein the 

selecting means is operable to compare the feature 
quantities of the same types among the plural key images 
specified by the specifying controller and wherein the 

5 determining means is operable to calculate an average value 
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of the feature quantities of the plural key image with 
respect to the types of the feature quantities selected by 
the selecting means, to thereby determine the calculated 
average value as representing the common feature quantities. 
5 14. An image searching method which comprises the 

steps of: 

storing a plurality of database images to be searched 
for in an image database; 

specifying a plurality of key images used to specify 
10 search conditions by means of a specifying controller; 

comparing by means of a calculating controller, the 
plural images, specified by the specifying controller, with 
the plural database images to thereby calculate 
similarities between the common feature values and the 
15 database image feature values; 

retrieving a particular key image from the specified 
images based on the similarities calculated by the 
calculating controller, by means of a selecting controller; 
and 

20 retrieving the images from the database images based 

on the similarity between the key image, selected by the 
selecting controller, and the database images, by means of 
a searching controller. 

15 * The method as claimed in Claim 14, wherein the 

25 selecting controller is operable to select as a particular 
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one of the plural specified key images, the key images 
which most resemble to the database images being searched 
for. 

16. The method as claimed in Claim 15, wherein the 

calculating controller is operable to calculate a plurality 
of types of the feature quantities from the plural key 
images and then to calculate a degree of similarity by 
comparing the feature quantities with the database images 
for each type, and wherein the selecting controller selects, 
as the particular key image from the plural specified 
images, the key images which most resemble to the database 
images being searched with respect to an average value of 
degrees of similarities calculated by the calculating means 
for each type of the feature quantities. 

17 • An image searching method which comprises the 

steps of: 

storing a plurality of database images to be searched 
for in an image database; 

specifying a plurality of key images used to specify 
search conditions by means of a specifying controller; 

comparing all of the key images, specified by the 
specifying controller, with respective feature values of 
the database images to thereby calculate similarities 
therebetween, by means of a first calculating controller; 

selecting by means of a second calculating controller, 
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a particular key image from the plural key images specified 
by the specifying controller and for comparing the 
particular key image with the database images to thereby 
calculate similarities therebetween; 

calculating, by means of a third calculating 
controller, a final similarity for use in searching based 
on the similarities calculated respectively by the first 
and second calculating controllers; and 

retrieving by means of a searching controller, one of 
the database images, which is similar to the particular key 
image, based on the final similarity calculated by the 
third calculating controller. 

18 • The image searching method as claimed in Claim 17, 

wherein the third calculating controller is operable to 
increase a weight of the degree of similarity, calculated 
by the first calculating controller, to a value greater 
than that of the degree of similarity, calculated by the 
second calculating controller, to thereby calculate the 
final degree of similarity. 

19 • Th e image searching method as claimed in claim 18, 

wherein the first calculating controller is operable to 
extract the common feature quantities of the image common 
to all of the key images, and to compare those common 
feature quantities with the database image to thereby 
5 calculate the degree of similarity. 



20 • The image searching method as claimed in Claim 19, 

wherein the second calculating controller is operable to 
select the key images most similar to the database image 
from the key images and to calculate the degree of 
similarity. 

21. A recording medium storing therein a computer- 

executable image searching program, said program 
comprising: 

storing a plurality of database images to be searched 
for in a database; 

specifying a plurality of key images used to specify 
search conditions by means of a specifying controller; 

extracting by means of an extracting controller, 
common feature values of common images from the plural key 
images specified by the specifying controller; 

comparing by means of a calculating controller, the 
common feature values, extracted by the extracting 
controller, with the feature values of the plural database 
images to thereby sequentially calculate similarities 
between the common feature values and the database image 
feature values; and 

retrieving from the database images one of the images 
which is similar to the key image, based on a similarity 
calculated by the calculating controller, by means of a 
searching controller. 
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22 • Th e recording medium as claimed in Claim 21, 

wherein the extracting controller includes an extracting 
means for extracting a plurality of types of the feature 
quantities from the respective key images specified by the 
specifying controller; a selecting means for comparing the 
feature quantities, extracted by the extracting sub-step, 
among the plural key images specified by the specifying 
controller to thereby select at least one of the types of 
the feature quantities; and a determining means for 
determining the common feature quantities based on the at 
least one type of the features quantities selected by the 
selecting means. 

23. The recording medium as claimed in Claim 22, 

wherein the selecting means is operable to compare the 

15 feature quantities of the same types among the plural key 
images specified by the specifying controller and wherein 
the determining means is operable to calculate an average 
value of the feature quantities of the plural key image 
with respect to the types of the feature quantities 

20 selected by the selecting means, to thereby determine the 
calculated average value as representing the common feature 
quantities . 

24 * A recording medium storing therein a computer- 

executable image searching program, said program 
25 comprising: 
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storing a plurality of database images to be searched 
for in an image database; 

specifying a plurality of key images used to specify 
search conditions by means of a specifying controller; 

comparing by means of a calculating controller, the 
plural images, specified by the specifying controller, with 
the plural database images to thereby calculate 
similarities between the common feature values and the 
database image feature values; 

retrieving a particular key image from the specified 
images based on the similarities calculated by the 
calculating controller, by means of a selecting controller; 
and 

retrieving the images from the database images based 
on the similarity between the key image, selected by the 
selecting controller, and the database images, by means of 
a searching controller, 

25. The recording medium as claimed in Claim 24, 
wherein the selecting controller is operable to select as a 
particular one of the plural specified key images, the key 
images which most resemble to the database images being 
searched for. 

26. The recording medium as claimed in Claim 25, 
wherein the calculating controller is operable to calculate 
a plurality of types of the feature quantities from the 
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plural key images and then to calculate a degree of 
similarity by comparing the feature quantities with the 
database images for each type, and wherein the selecting 
controller selects, as the particular key image from the 
plural specified images, the key images which most resemble 
to the database images being searched with respect to an 
average value of degrees of similarities calculated by the 
calculating means for each type of the feature quantities. 
27 . A recording medium storing therein a computer- 

executaJDle image searching program, said program 
compri sing : 

storing a plurality of database images to be searched 
for in an image database; 

specifying a plurality of key images used to specify 
search conditions by means of a specifying controller; 

comparing all of the key images, specified by the 
specifying controller, with respective feature values of 
the database images to thereby calculate similarities 
therebetween, by means of a first calculating controller; 

selecting by means of a second calculating controller, 
a particular key image from the plural key images specified 
by the specifying controller and for comparing the 
particular key image with the database images to thereby 
calculate similarities therebetween; 

calculating, by means of a third calculating 
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controller, a final similarity for use in searching based 
on the similarities calculated respectively by the first 
and second calculating controllers; and 

retrieving by means of a searching controller, one of 
the database images, which is similar to the particular key 
image, based on the final similarity calculated by the 
third calculating controller. 

28 • Th e recording medium as claimed in Claim 27, 
wherein the third calculating controller is operable to 
increase a weight of the degree of similarity, calculated 
by the first calculating controller, to a value greater 
than that of the degree of similarity, calculated by the 
second calculating controller, to thereby calculate the 
final degree of similarity. 

29 • Th e recording medium as claimed in claim 28, 
wherein the first calculating controller is operable to 
extract the common feature quantities of the image common 
to all of the key images, and to compare those common 
feature quantities with the database image to thereby 

0 calculate the degree of similarity. 

30 • The recording medium as claimed in Claim 29, 
wherein the second calculating controller is operable to 
select the key images most similar to the database image 
from the key images and to calculate the degree of 

5 similarity. 
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Abstract of the Disclosure 

An image searching system, an image searching 
method, and a medium storing an image searching program for 
improving search accuracy while reducing the user burden 
when searching for similar images based on a plurality of 
key images are disclosed. 

A plurality of feature quantity types is 
extracted from each of plural key images (S42), and feature 
quantities of the same type are compared between the key 
images to select those feature quantities having 
approximately equal values as the key image feature 
quantities to be used as search criteria (S43) . Feature 
quantities in the target images are then extracted (S45) 
and a degree of similarity between the target image and key 
images is calculated based on a key image feature quantity 
and the same feature quantity in the target image (S4 6) . 
Based on the resulting degree of similarity, target images 
having a high similarity to the key image are output as the 
search result (S47) . 
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DECLARATION AND POWF.P OF ATTOttNFV 

As a below-named inventor, I hereby declare that: 

My residence, post office address and citizenship are 
as stated below next to my name; 

I believe that I and the other joint inventors named 
below are the original, first and joint inventors of the 
subject matter which is claimed and for which a patent is 
sought on the invention or discovery entitled 

AN IMAGE SEARCHING SYSTEM AND IMAGE SEARCHING METHOD, AND 
A RECORDING MEDIUM STORING AN IMAGE SEARCHING PROGRAM 



the specification of which (X) is attached hereto; 
( ) was filed on t as U.S. Ser. No. 



I have reviewed and understand the contents of the 
above -identified specification, including the claims; and 

I acknowledge the duty to disclose to the Patent and 
Trademark Office all information known to me which is 
material to patentability as defined in 37 C.F.R. § 1.56. 

I hereby claim foreign priority benefits under 
35 U.S.C. § 119/3 65 of any foreign application for patent or 
inventor's certificate as listed below or of any PCT 
international application, designating at least one country 
other than the United States of America, as listed below and 
have also identified below any foreign application for 
patent or inventor's certificate or any PCT international 
application, designating at least one country other than the 
United States of America, directed to said invention or 



2 

discovery and having a filing date before that of the 
applications on which priority is claimed: 

fflZMBEB COUNTRY PATE FILED PRIORITY CLAIMED 

(Yes) (No) 

10-41989 Japan February 24,. 1998 x 



I hereby appoint: 

Dale B. Nixon, Reg. No. 28 , 454 
William R. Gustavson, Reg. No. 29,160 
David L. Hitchcock, Reg. No. 30,067 
Roger N. Chauza, Reg. No. 29,753 
Eugenia S. Hansen, Reg. No. 31,966 
James W. Williams, Reg. No. 20,047 
Steven P. Rhines, Reg. No. 38,595 
Thomas N. Tarnay, Reg. No. 41,341 
Daren C. Davis, Reg. No. 38,425 



all of the firm of SIDLEY & AUSTIN, my representatives with 
full power of substitution and revocation, to prosecute this 
application and to transact all business in the United 
States Patent and Trademark Office connected therewith, and 
to file and prosecute any international patent applications 
filed thereon before any international authorities under the 
Patent Cooperation Treaty. 



Send correspondence to: 
SIDLEY & AUSTIN 
717 N. Harwood 
Suite 3400 

Dallas, Texas 75201-6507 



Direct telephone calls to: 

James Wt Williams 

Direct: (214) 981-3328 

Main: (214) 981-3300 

Attorney Docket No. : 



I hereby declare that all statements made herein of my 
own knowledge are true and that all statements made on 



information and belief are believed to be true; and further 
that these statements were made with the knowledge that 
willful false statements and the like so made are punishable 
by fine or imprisonment, or both, under Section 1001 of 
Title 18 of the United States Code, and that such willful 
false statements may jeopardize the validity of the 
application or any patent issuing thereon. 



Full name of first joint inventor: 



Sumivo TANAKA 

First Middle Last 

Inventor 1 s signature: ^yl/h'i^tp ^o3asasI<£&^ 

Date: P&J? /T. 9 

Residence: Osaka-Shi, Osaka, Japan 



Citizenship : Japan 

Post Office Address: c/o MINOLTA Co., Ltd., Osaka Kokusai Building, 
3-13, 2-chome, Azuchi-machi , Chuo-ku, Osaka-shi, Osaka 541-8556, Japan 



Full name of second joint inventor: 



First Middle Last 

Inventor's signature: 

Date: 

Residence : 



Citizenship: 

Post Office Address: 



